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Abstract. This paper deals with chain graphs under the Andersson-Madigan-Pcrlman 
(AMP) interpretation. In particular, we present a constraint based algorithm for learning 
an AMP chain graph a given probability distribution is faithful to. Moreover, we show that 
the extension of Meek's conjecture to AMP chain graphs does not hold, which compromises 
the development of efficient and correct score+search learning algorithms under assumptions 
weaker than faithfulness. 

We also introduce a new family of graphical models that consists of undirected and bidi- 
rected edges. We name this new family maximal covariance-concentration graphs (MCCGs) 
because it includes both covariance and concentration graphs as subfamilies. However, every 
MCCG can be seen as the result of marginalizing out some nodes in an AMP CG. We describe 
global, local and pairwise Markov properties for MCCGs and prove their equivalence. We 
characterize when two MCCGs are Markov equivalent, and show that every Markov equiva- 
lence class of MCCGs has a distinguished member. We present a constraint based algorithm 
for learning a MCCG a given probability distribution is faithful to. 

Finally, we present a graphical criterion for reading dependencies from a MCCG of a prob- 
ability distribution that satisfies the graphoid properties, weak transitivity and composition. 
We prove that the criterion is sound and complete in certain sense. 



1. Introduction 



This paper d eals with chain graphs ( CGs) under the Andersson-Madigan-Perlman (AMP) 
interpretation ( Andersson et al. . 200lh . Two other interpretatio ns exist in the li terature, 
namely the Lauritzen-Wermuth-Frydenberg (L WF') interpretation (lLauritzenl . Il996l ) and the 
multivariate regression (MVR) interpretation ( Cox and Wermuth . 19961 ). The AMP and 
LWF interpretations ar e sometimes cons i dered as competing and, thus, t heir relative merits 



Roverato and Studenyi . 



have been pointed out ( Andersson et al.l . l2001t iDrton and Eichlerl . l2006r iLevitz et al.l . 12001 



20061 ). Note, however, that no interpretation subsumes the other: 



There are many independence models that can be induced by a CG u nder one interpretation 



but that cannot be induced by any CG under the other interpretation (lAndersson et al.l . 12001 



Theorem 6 ). Likewise, neither the A MP interpretation subsumes the MVR interpretation nor 
vice versa ( Sonntag and Pefial . 2013 . Theorems 4 and 5). 

This paper consists of three main sections. In Section |3l we present an algorithm for learn- 
ing an AMP CG a given probability distribution is faithful to. To our knowledge, we are 
the first to present such an algorithm. However, algorith ms for learning LWF CGs under 
faithfulness already exist ( Ma et aD . 2008 ; Studeny . 1997a ). In fact, we have recently devel- 
oped an algorithm for learning LWF CGs under the milder composition property assumption 
( Pena et al.l . 2012 ). We have also recently developed an al gorithm for learning MVR CGs 
under the faithfulness assumption ( ISonntag and Penal . 120121 ) . 

In Section HJ we introduce a new family of graphical models that consists of undirected and 
bidirected edges. We name this new family maximal covariance-concentration graphs (MC- 
CGs) because it includes both covariance and concentration graphs as subfamilies. However, 
every MCCG can be seen as the result of marginalizing out some nodes in an AMP CG. 
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Concentration graphs (a.k.a Markov networks) were introduced bv lPearll (119881 ) to represent 
independence models. Specifically, the concentration graph of a probability distribution p is 
the undirected graph G where two nodes are not adjacent if and only if their corresponding 
random variables are independent in p given the rest of the random variables. Graphical cri- 
teria for reading depen dencies and independencies from G (under certa in assumptions about 
p) have been proposed ( Bouckaertl . 1995 ; Pearl . 1988 ; Pena et al. L 20091). L ikewi se, covariance 
graphs (a.k.a bidirected graphs) were introduced by Cox and Wermuthl f 19961 ) to represent 
independence models. Specifically, the covariance graph of a probability distribution p is 
the bidirected graph G where two nodes are not adjacent if and only if their corresponding 
random variables are marginally independent in p. Graphical criteria for reading dependen- 
cies and independencies from G (under certain as s umptions a bout p) have been proposed 
( Banerjee and Richardson . 20031 ; Kauermann . 1996 ; |PenaLl2013h . 

If we focus on Gaussian probability distributions, then one could say that the covariance 
graph of a Gaussian probability distribution models its covariance matrix, whereas its con- 
centration graph models its concentration matrix. We think that Gaussian probability dis- 
tributions would be modeled more accurately if their covariance and concentration matrices 
were modeled jointly by a single graph. It is for this reason that we develop MCCGs. 

Finally, in Section[5]we present a graphical criterion for reading dependencies from a MCCG 
G of a probability distribution p, under the assumption that G satisfies some topological 
constraints and p satisfies the graphoid properties, weak transitivity and composition. We 
prove that the graphical criterion is sound and complete in certain sense. 



2. Preliminaries 

In this section, we review some concepts from probabilistic graphical models that are used 
later in this paper. All the graphs and probability distributions in this paper are defined over 
a finite set V. All the graphs in this paper are simple, i.e. they contain at most one edge 
between any pair of nodes. The elements of V are not distinguished from singletons. We 
denote by |X| the cardinality of X £ V. 

If a graph G contains an undirected, directed or bidirected edge between two nodes V± and 
V2, then we write that V1-V2, V\-> V2 or Vi <->■ V2 is in G. The parents of a set of nodes X 
of G is the set pac(X) = {Vi\Vi ->■ V2 is in G, V\ i X and V2 e X}. The neighbors of a set 
of nodes X of G is the set nec(X) = {Vi|Vi - V 2 is in G, V\ £ X and V 2 e X}. The spouses 
of a set of nodes X of G is the set spc(X) = {Vi|Vi <-> V 2 is in G, V\ i X and V 2 e X}. The 
adjacents of a set of nodes X of G is the set adc(X) = {Vi|Vi ->• V 2 , V\ - V2 or V\ <- V 2 is 
in G, V\ £ X and V2 e X}. A route from a node Vi to a node V n in G is a sequence of (not 
necessarily distinct) nodes Vi, . . . , V n such that Vi e adciVi+i) for all 1 < i < n. If the nodes in 
the route are all distinct, then the route is called a path. The length of a route is the number 
of (not necessarily distinct) edges in the route, e.g. the length of the route V\, . . . , V n is n- 1. 
A route is called a cycle if V n = V\. A route is called descending if Vi e paaiVi+x) u neciVi+i) 
for all 1 < i < n. The descendants of a set of nodes X of G is the set dec(X) = {V n \ there is 
a descending route from V\ to V n in G, V% e X and V n i X}. A cycle is called a semidirected 
cycle if it is descending and Vi -> Vi+i is in G for some 1 < i < n. A chain graph (CG) is a 
graph whose every edge is undirected or directed, and that has no semidirected cycles. A set 
of nodes of a graph is complete if there is an undirected edge between every pair of nodes in 
the set. A set of nodes of a graph is undirectly (respectively bidirectly) connected if there 
exists a route in the graph between every pair of nodes in the set such that all the edges in 
the route are undirected (respectively bidirected). An undirected (respectively bidirected) 
connectivity component of a graph is an undirectly (respectively bidirectly) connected set 
that is maximal (with respect to set inclusion). The undirected connectivity component a 
node A of a graph G belongs to is denoted as coq(A). The subgraph of G induced by a set of 
its nodes X, denoted as Gx, is the graph over X that has all and only the edges in G whose 
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both ends are in X. An immorality in a CG is an induced subgraph of the form A -> B <- C '. 
A flag in a CG is an induced subgraph of the form A -> 5 - C. If a CG (7 has an induced 
subgraph of the form A ^ B C or A ^ B - C , then we say that the triplex ({A, C},B) is 
in G. Two CGs are triplex equivalent if and only if they have the same adjacencies and the 
same triplexes. 

Let X, Y, Z and W denote four disjoint subsets of V. An independence model M is a 
set of statements X 1 mY\Z. M satisfies the graphoid properties if it satisfies the following 
properties: Symmetry X l mY\Z =>Yl mX\Z, decomposition X l mY u W\Z => X 1 mY\Z, 
weak union Xl M Y u W\Z =>Il M Y\Z u W, contraction Xl M Y\Z u W aX± m W\Z =>Il 
M Y u W\Z, and intersection X l p Y\Z uW a X l p W\Z uY => X ± P Y u W\Z. Moreover, M 
satisfies the composition property if X 1 M Y\Z a X 1 mW\Z => X 1 M Y u Finally, M 

satisfies the weak transitivity property if X l M Y\Z a X ± M Y\Z u K => Xl mK\Z v K ± m Y\Z 
with Xel/\X\y\Z. We say that an independence model is a WTC graphoid when 
it satisfies the seven previous properties. We denote by X l p Y\Z (respectively X J_ p Y\Z) 
that X is independent (respectively dependent) of Y given Z in a probability distribution 
p. We say that p is Markovian with respect to an independence model M when X l p Y\Z if 
X l mY\Z for all X, Y and Z disjoint subsets of V. We say that p is faithful to M when 
if and only if X ±mY\Z for all X, Y and Z disjoint subsets of V. Any probability 
distribution p satisfies the first four previous properties. If p is faithful to a CG, then it also 
satisfies the last three previous properties^ 

A node B in a route p in a CG is called a head-no-tail node in p if A -> 5 •«- C, ^4 -> 5 - C, 
or A - <- C is a subroute of p (note that maybe A = C in the first case). Let X, Y and Z 
denote three disjoint subsets of V. A route p in a CG C7 is said to be Z-open when (i) every 
head-no-tail node in p is in Z, and (ii) every other node in p is not in Z. When there is no 
route in G between a node in X and a node in Y that is Z-open, we say that X is separated 
from Y given Z in G and denote it as X l g y|Z0 We denote by XJ_ G Y\Z that Xl G y|Z does 
not hold. The independence model induced by G, denoted as 1(G), is the set of separation 
statements X l G Y\Z. If two CGs G and H are triplex equivalent, then 1(G) = I(H)u 



3. Algorithm for Learning AMP CGs 



In this section, we present an algorithm for learning an AMP CG a given probability 
distribution is faithful to. The algorithm, which can b e seen in Table [H resembles the well- 



known PC algorithm (iMeekl . 119951 iSpirtes et al.l . Il993l ). It consists of two phases: The first 
phase (lines 1-8) aims at learning adjacencies, whereas the second phase (lines 9-10) aims at 
directing some of the adjacencies learnt. Specifically, the first phase declares that two nodes 
are adjacent if and only if they are not separated by any set of nodes. Note that the algorithm 
does not test every possible separator (see line 5). Note also that the separators tested are 
tested in increasing order of size (see lines 2, 5 and 8). The second phase consists of two steps. 
In the first step, the ends of some of the edges learnt in the first phase are blocked according 
to the rules R1-R4 in Table |2j A block is represented by a perpendicular line such as in i— 
or i i , and it means that the edge cannot be directed in that direction. In the second step, 
the edges with exactly one unblocked end get directed in the direction of the unblocked end. 
The rules R1-R4 work as follows: If the conditions in the antecedent of a rule are satisfied, 



see it, note that there is a Gaussian distribution that is faithful to G ( Levitz et al.l . l200lL Theorem 6.1). 
Moreover , every Gaussian distribution satisfies the intersection, composition and weak transitivity properties 
( Studenvl . 20051. Propositi o n 2.1 and Corollaries 2.4 and 2.5). 

2 See (jAndersson et al. . 2001 , Remark 3.1) for the equivalence of this and the standard definition of 
separation. 

3 To see it, note that there are Gaussian distributions p and q that are faithful to G and H, respectively 
( Levitz et al.l. 2001 . Theor em 6.1). Moreov er, p and q are Mar kovian with respect to H and G, respectively, 
by I Andersson et al.l (|200l[ Theorem 5) and lLevitz et al.l (|200lL Theorem 4.1). 
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Table 1. Algorithm for learning AMP CGs. 



Input: A probability distribution p that is faithful to an unknown CG G. 
Output: A CG H that is triplex equivalent to G. 

1 Let H denote the complete undirected graph 

2 Set I = 

3 Repeat while I < \V\ - 2 

4 For each ordered pair of nodes A and B in H st A e adn(B) and |[ad//(^4) u ad if (ad ff ( J 4))] \ B| > / 

5 If there is some S £ [ad H (A) u ad H (ad H (A))] \ B such that |5| = I and A± p B\S then 

6 Set Sab = S B a = S 

7 Remove the edge A - B from H 

8 Set I = I + 1 

9 Apply the rules R1-R4 to H while possible 
10 Replace every edge >— (»— i) in if with -> (-) 



Table 2. Rules R1-R4 in the algorithm for learning AMP CGs. 



Rl: 0B0 oC => ^ 0B0 \C 

aB{S ac 

R2: A i oBo oC => ^ 0B1 oC 

a B s S AC 



c c 

s \ / \ 

R4: A o o 5 ,4 i o B 



D D 

A ,4 € S C D 



then the modifications in the consequent of the rule are applied. Note that the ends of some 
of the edges in the rules are labeled with a circle such as in i— ° or The circle represents 
an unspecified end, i.e. a block or nothing. The modifications in the consequents of the rules 
consist in adding some blocks. Note that only the blocks that appear in the consequents are 
added, i.e. the circled ends do not get modified. The conditions in the antecedents of Rl, R2 
and R4 consist of an induced subgraph of H and the fact that some of its nodes are or are not 
in some separators found in line 6. The condition in the antecedent of R3 is slightly different 
as it only says that there is a cycle in H whose edges have certain blocks, i.e. it says nothing 
about the subgraph induced by the nodes in the cycle or whether these nodes belong to some 
separators or not. Note that, when considering the application of R3, one does not need to 
consider intersecting cycles, i.e. cycles containing repeated nodes other than the initial and 
final ones. 



3.1. Correctness of the Algorithm. In this section, we prove that our algorithm is correct, 
i.e. it returns a CG the given probability distribution is faithful to. We start proving a result 
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for any probability distribution that satisfies the intersection and composition properties. 
Recall that any probability distribution that is faithful to a CG satisfies these properties and, 
thus, the following result applies to it. 

Lemma 1. Let p denote a probability distribution that satisfies the intersection and compo- 
sition properties. Then, p is Markovian with respect to a CG G if and only if p satisfies the 
following conditions: 

CI: A± p co G (A) \ A \ ne G (A)\pa G (A u ne G (A)) u ne G (A) for all AeV, and 
C2: Al p V \ A \ de G (A) \ pa G (A)\pa G (A) for all AeV. 

Proof. It follows from Andersson et al.l ( 200ll . Theorem 3) and Levitz et al. ( 2001 . Theorem 



4.1) that p is Markovian with respect to G if and only if p satisfies the following conditions: 

LI: A± p co G (A) \ A \ ne G (A)|[V \ co G {A) \ de G {co G {A))] u ne G {A) for all AeV, and 
L2: AlpV \ co G (A) \ de G (co G (A)) \ pa G (A)\pa G (A) for all A e V. 

Clearly, C2 holds if and only if L2 holds because de G (A) = [co G (A) u de G (co G (A))] \ A. 
We prove below that if L2 holds, then CI holds if and only if LI holds. We first prove the if 
part. 

1. Bl p V \ co G {B) \ de G {co G {B)) \ pa G (B)\pa G (B) for all B e A u ne G {A) by L2. 

2. Bl p V \ co G (B) \ de G (co G (B)) \ pa G (A u ne G (A))\pa G (A u ne G (A)) for all B e A u 
ne G (A) by weak union on 1. 

3. A u ne G (A) 1 P V \ co G (A) \ de G (co G (A)) \ pa G (A u ne G (A))\pa G (A u ne G (A)) by 
repeated application of symmetry and composition on 2. 

4. Al p V \ co G (A) \ de G (co G (A)) \ pa G (A u ne G (A))\pa G (A u ne G {A)) u ne G (A) by 
symmetry and weak union on 3. 

5. Al p co G {A) \ A \ ne G (A)\[V \ co G {A) \ de G {co G {A))] u ne G {A) by LI. 

6. Al p [co G (A) \A\ ne G (A)] u [V \ co G (A) \ rfeG(co G (^)) \pac(^4 u ne G (v4))]|pa G (A u 
ne G (v4)) u ne G (^4) by contraction on 4 and 5. 

7. Al p co G (A) \ A \ ne G (A)\pa G (A u ne G (A)) u ne G (^4) by decomposition on 6. 

We now prove the only if part. 

8. AlpCo G (A) \A\ ne G (A)\pa G (A u ne G (A)) u ne G (A) by CI. 

9 . A l p [V \ co G ( A) \ de G (co G ( A) ) \ pa G ( A u ne G ( A) ) ] u [co G ( A) \ A \ ne G ( A) ] |pa G ( A u 
rze G (A)) u ne G (A) by composition on 4 and 8. 

10. A± p co G (y4) \ A \ ne G (A)|[V A \ co G (A) \ c/e G (co G (A))] u ne G (A) by weak union on 9. 

□ 

Lemma 2. After line 8, G and H have the same adjacencies. 

Proof. Consider any pair of nodes A and B m G. If A e ad G (B), then A / p B\S for all 
S c V \ [A u B] by the faithfulness assumption. Consequently, A e adn(B) at all times. On 
the other hand, if A i ad G (B), then consider the following cases. 

Case 1: Assume that co G (A) = co G (B). Then, A l p co G (A) \ A \ ne G (A)\pa G (A u 
ne G (A))une G (A) by CI in Lemma [Hand, thus, Al p B\pa G (Au ne G (A)) u ne G (A) by 
decomposition and B { ne G (A), which follows from A { ad G (B). Note that, as shown 
above, pa G (A u ne G (A)) u ne G (A) c [adn(A) u adjf(a<ijj(A))] \ B at all times. 

Case 2: Assume that co G (A) * co G (B). Then, A ^ de G (B) oy B $. de G (A) because G 
has no semidirected cycle. Assume without loss of generality that B £ de G (A). Then, 
Alp V \ A \ de G (A) \ pa G (A)\pa G (A) by C2 in Lemma [Hand, thus, Al p B\pa G (A) 
by decomposition, B £ de G (A), and B i pa G (A) which follows from A $ ad G (B). Note 
that, as shown above, pa G (A) £ adu(A) \ B ai all times. 

Therefore, in either case, there will exist some S in line 5 such that Al p B\S and, thus, the 
edge A - B will be removed from H in line 7. Consequently, A i adn(B) after line 8. □ 



6 



The next lemma proves that the rules R1-R4 are sound, i.e. if the antecedent holds in G, 
then so does the consequent. 

Lemma 3. The rules R1-R4 are sound. 

Proof. According to the antecedent of Rl, G has a triplex ({A,C}, B). Then, G has an 
induced subgraph of the form A^B-^C,A^B-C or A - B ^ C. In either case, the 
consequent of Rl holds. 

According to the antecedent of R2, (i) G does not have a triplex ({A,C} , B), (ii) A^ B 
or A - B is in G, (hi) B e ad G (C), and (iv) A i ad G {C). Then, B -> C or B - C is in G. In 
either case, the consequent of R2 holds. 

According to the antecedent of R3, (i) G has a descending route from A to B, and (ii) 
A e adc(B). Then, A -> B or A - B is in G, because G has no semidirected cycle. In either 
case, the consequent of R3 holds. 

According to the antecedent of R4, neither B -> C nor B -> D are in G. Assume to the 
contrary that A <- B is in G. Then, G must have an induced subgraph that is consistent with 



C 




D 



because, otherwise, it would have a semidirected cycle. However, this induced subgraph 
contradicts that Ae Scd- D 

Lemma 4. After line 10, G and H have the same triplexes. Moreover, H has all the im- 
moralities in G. 

Proof. We first prove that any triplex in H is in G. Assume to the contrary that H has a 
triplex ({A,C}, B) that is not in G. This is possible if and only if, when line 10 is executed, 
H has an induced subgraph of one of the following forms: 

A — B — i C A^B^C A^-iB—iC 



where B e Sac by Lemma [2] The first and second forms are impossible because, otherwise, 
A^ B would be in H by R2. The third form is impossible because, otherwise, B <—< C would 
be in H by R2. 

We now prove that any triplex ({A,C},B) in G is in H. Let the triplex be of the form 
A -*■ B <- C '. Then, when line 10 is executed, A i— q B °— ■ C is in H by Rl, and neither A<—<B 
nor B i i C is in IP by Lemmas [2] and [3J Then, the triplex is in H. Note that the triplex is 
an immorality in both G and H. Likewise, let the triplex be of the form A -> B - C ' . Then, 
when line 10 is executed, A h- o B <>-< C is in H by Rl, and A <— i B is not in H by Lemmas 
[2] and [3l Then, the triplex is in H. Note that the triplex is a flag in G but it may be an 
immorality in H. □ 

Lemma 5. After line 9, H does not have any induced subgraph of the form A^oB^c 

Proof. Assume to the contrary that the lemma does not hold. Consider the following cases. 
Case 1: Assume that A i— o B is in H due to Rl. Then, when Rl was applied, H had 
an induced subgraph of one of the following forms: 

Ah^B^C A^B^C 

I IS 

D D 



case 1.1 



case 1.2 
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Case 1.1: If B i S CD then B -> C is in H by Rl, else B h- C is in if by R2. Either 

case is a contradiction. 
Case 1.2: If C £ S AD then A h- C is in H by Rl, else B -< C is in # by R4. Either 
case is a contradiction. 
Case 2: Assume that A h- ° B is in £T due to R2. Then, when R2 was applied, H had 
an induced subgraph of one of the following forms: 



Ah^B — C A^B — C A^B—^C A^B — C 
1 

D D D 1 zr . 

case 2.1 case 2.2 case 2.3 case 2.4 



Case 2.1: If A £ S CD then A -t C is in H by Rl, else A h- C is in 7? by R2. Either 

case is a contradiction. 
Case 2.2: Restart the proof with D instead of A and A instead of B. 
Case 2.3: Then, A — i C is in H by R3, which is a contradiction. 
Case 2.4: If (7 £ S BD then B *- C is in H by Rl, else B -. C is in if by R2. Either 
case is a contradiction. 
Case 3: Assume that A i— o S is in if due to R3. Then, when R3 was applied, ii had 
an induced subgraph of one of the following forms: 




case 3.1 case 3.2 case 3.3 case 3.4 



Note that C cannot belong to the route Aw...i-»D because, otherwise, A ^ C 
would be in H by R3. 

Case 3.1: If B i S CD then B C is in H by Rl, else B ■- C is in # by R2. Either 

case is a contradiction. 
Case 3.2: Restart the proof with D instead of A. 
Case 3.3: Then, B — i C is in if by R3, which is a contradiction. 
Case 3.4: Then, A \— C is in H by R3, which is a contradiction. 
Case 4: Assume that A B is in if due to R4. Then, when R4 was applied, B had 
an induced subgraph of one of the following forms: 



Ah>B-C Ah^B — C Ai-oB — C A^B — C 




case 4.1 case 4.2 case 4.3 case 4.4 



Cases 4.1-4.3: If B i S CD or B £ S CE then B -> C is in H by Rl, else B h- C is in 
H by R2. Either case is a contradiction. 

Case 4.4: Assume that C e Sde- Then, B — i C is in ii by R4, which is a contra- 
diction. On the other hand, assume that C i Sde- Then, it follows from applying 
Rl that B has an induced subgraph of the form 
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Note that A e Sde because, otherwise, R4 would not have been applied. Then, 
A \— C is in H by R4, which is a contradiction. 

□ 

Lemma 6. After line 9, every cycle in H that has an edge also has an edge — i. 

Proof. Assume to the contrary that H has a cycle p ■ V\ , . . . , V n = V\ that has an edge >— but 
no edge — i. Note that every edge in p cannot be h-o because, otherwise, every edge in p would 
be i ' by repeated application of R3, which contradicts the assumption that p has an edge i— . 
Therefore, p has an edge - or — i. Since the latter contradicts the assumption that the lemma 
does not hold, p has an edge -. Assume that p is of length three. Then, p is of one of the 
following forms: 

v 1 Cv7^v 3 v 1 'Cv^v 3 v 1 Cv7^v z _ 

The first form is impossible by Lemma The second form is impossible because, otherwise, 
V2 — 1 V3 would be in H by R3. The third form is impossible because, otherwise, V\ >— V3 
would be in H by R3. Thus, the lemma holds for cycles of length three. 

Assume that p is of length greater than three. Recall from above that p has an edge - and 
no edge — 1. Let Vi+\ - V- l+ 2 be the first edge - in p. Assume without loss of generality that 
i > 0. Then, p has a subpath of the form Vi >-° Vi+\ - Vi+2- Note that Vi e aduiVi^) because, 
otherwise, if V + i i SviV i+2 then V M — 1 V i+2 would be in H by Rl, else V M >— V i+2 would be in 
H by R2. Thus, H has an induced subgraph of one of the following forms: 

Vi wV^- Vj +2 Vi "wV^ 1 - V i+2 V ^Vm~- v i+2 , 

The first form is impossible by Lemma [5j The second form is impossible because, otherwise, 
Vi+i 1 Vi + 2 would be in H by R3. Thus, the third form is the only possible. Note that this 
implies that g : V\, . . . , Vi, Vi +2 , ■ ■ ■ , V n = V\ is a cycle in H that has an edge 1— and no edge — 1. 

By repeatedly applying the reasoning above, one can see that H has a cycle of length three 
that has an edge 1— and no edge — 1. As shown above, this is impossible. Thus, the lemma 
holds for cycles of length greater than three too. □ 

Theorem 1. After line 10, H is triplex equivalent to G and it has no semidirected cycle. 

Proof. Lemma [2] implies that G and H have the same adjacencies. Lemma H] implies that G 
and H have the same triplexes. Lemma [6] implies that H has no semidirected cycle. □ 

3.2. Discussion. In this section, we have presented an algorithm for learning an AMP CG 
a given probability distribution p is faithful to. In practice, of course, we do not usually have 
access to p but to a finite sample from it. Our algorithm can easily be modified to deal with 
this situation: Replace A± P B\S in line 5 with a hypothesis test, preferably with one that is 
consistent so that the resulting algorithm is asymptotically correct. 

It is worth mentioning that, whereas Rl, R2 and R4 only involve three or four nodes, R3 
may involve many more. Hence, it would be desirable to replace R3 with a simpler rule such 
as 

Unfortunately, we have not succeeded so far in proving the correctness of our algorithm 
with such a simpler rule. Note that the output of our algorithm will be the same whether we 
keep R3 or we replace it with a simpler sound rule. The only benefit of the simpler rule may 
be a decrease in running time. 

We have shown in LemmaH]that, after line 10, H has all the immoralities in G or, in other 
words, every flag in H is in G. The following lemma strengthens this fact. 
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Lemma 7. After line 10, every flag in H is in every CG F that is triplex equivalent to G. 



Proof. Note that every flag in H is due to an induced subgraph of the form A <— B i— i C. 
Note also that all the blocks in H follow from the adjacencies and triplexes in G by repeated 
application of R1-R4. Since G and F have the same adjacencies and triplexes, all the blocks 
in H hold in both G and F by Lemma [3J □ 



The lemma above implies t hat, in terms of IRoverato and Studenyl (120061 ). our algorithm 



outputs a deflagged graph. IRoverato and Studenyl ( 120061 ) also introduce the concept of 



strongly equivalent CGs: Two CGs are strongly equivalent if and only if they have the same 
adjacencies, immoralities and flags. Unfortunately, not every edge ->■ in H after line 10 is in 
every deflagged graph that is triplex equivalent to G, as the following example illustrates, 
where both G and H are deflagged graphs. 



A B 



C 



B 


A 


B 


I\ 
D — E 


I 

C 


1 

-D 


G 




H 



■E 



Therefore, in terms of IRoverato and Studenv (2006). our algorithm outputs a deflagged 
graph but not the largest deflagged graph. The latter is a distinguished member of a class of 
triplex equivalent CGs. Fortunat ely, the largest deflagge d grap h can easily be obtained from 
any deflagged graph in the class ( Roverato and Studenv . 20061 . Corollary 17). 

The correctness of our algorithm lies upon the assumption that p is faithful to some CG. 
This is a strong requirement that we would like to weaken, e.g. by replacing it with the milder 
assumption that p satisfies the composition property. Correct algorithms for learning directed 
and a cyclic graphs (a.k.a. Bayesian networks) under the composition property assumption 
exist (jChickering and Meek! . |2002| ; iNielsen et al.l . 120031 ) . We have recently develope d a correct 
algorithm for learning LWF CGs under the composition property f Pena et all 20121 ). The way 
in which these algorithms proceed (a.k.a. score+search based approach) is rather different 
from that of the algorithm presented in this section (a.k.a. constraint based approach). In 
a nutshell, they can be seen as consisting of two phases: A first phase that starts from the 
empty graph H and adds single edges to it until p is Markovian with respect to H, and a 
second phase that removes single edges from H until p is Markovian with respect to H and 
p is not Markovian with respect to any CG F such that 1(H) c 1(F). The success of the 
first phase is guaranteed by the composition property assumption, wher e as the success of the 
second phase is guaranteed by the so-called Meek's conjecture dMeekl . Il997t ). Specifically, 
given two directed and acyclic graphs F and H such that 1(H) c 1(F), Meek's conjecture 
states that we can transform F into H by a sequence of operations st, after each operation, 
F is a directed and acyclic graph and 1(H) c 1(F). The operations consist in adding a 
single edge to F, or replacing F with a triplex equival ent directed and acyclic graph. Meek's 
conjecture was proven to be true in ( Chickerind . 20021. Theorem 4). The extension of Meek's 
conjecture to LWF CGs was proven to be true in (jPenal . 1201 lk Theorem 1). Unfortunately, 
the extension of Meek's conjecture to AMP CGs does not hold, as the following example 
illustrates. 

Example 1. Consider the AMP CGs F and H below. 



A B 



A B 



C—D—E C—D—E 
F H 



Then, 1(H) = {Xl H Y\Z :Xl H Y\Z e I X (H) u I 2 (H) v Y 1„X\Z e h(H) u I 2 (H)} where 
h(H) = {A± H Y\Z :Y,Zc BuCuEaD { Z} and I 2 (H) = {C± H Y\Z : Y c BuEaAuD c Z}. 
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One can easily confirm that 1(H) £ 1(F) by using the definition of separation. However, there 
is no CG that is triplex equivalent to F or H and, obviously, one cannot transform F into H 
by adding a single edge. 

While the example above compromises the development of score+search learning algorithms 
that are correct and efficient under the composition property assumption, it is not clear to us 
whether it also does it for constraint based algorithms. This is something we plan to study. 

4. Maximal Covariance- Concentration Graphs 

In this section, we introduce a new family of graphical models. This new family generalizes 
both covariance and concentration graphs, and each of its members can be seen as the result of 
marginalizing out some nodes in an AMP CG. Specifically, we define covariance-concentration 
graphs (CCGs) as graphs whose every edge is undirected or bidirected. A node B in a path p 
in a CCG is called a triplex node in p if A B C, A++B-C or A-B++C is & subpath 
of p. Let X, Y and Z denote three disjoint subsets of V. A path p in a CCG G is said to be 
Z-open when 

• every triplex node in p is in Z, and 

• every non-triplex node in p is not in Z or has some spouse in G. 

When there is no path in G between a node in X and a node in Y that is Z-open, we say 
that X is separated from Y given Z and denote it as X l G Y\Z. We denote by X J_ G Y\Z that 
X l G Y\Z does not hold. The independence model induced by G is the set of separations 
Xl G Y\Z. 

Typically, every missing edge in a graphical model corresponds to a separation. However, 
this is not true for CCGs. For instance, the CCG G below does not contain any edge between 
B and D but B ]_ G D\Z for all Z c V\{B, D}. Likewise, G does not contain any edge between 
A and E but AJ_ G E\Z for all Z c V \ {A, E). 

1 

F 

In order to avoid the problem above, we focus in this paper on what we call maximal CCGs 
(MCCGs), which are those CCGs that have 

• no induced subgraph A - C - B such that C has some spouse, and 

• no cycle A - ... - B *-*■ A. 

Hereinafter, we refer to the two constrains on CCGs above as CI and C2, respectively. As 
Theorem [3] shows, every missing edge in a MCCG corresponds to a separation. So, no edge 
can be added to a MCCG without changing the independence model induced by it, hence the 
name. Note that both covariance and concentration graphs are MCCGs, and the definitions 
of separation for covariance and concentration graphs are special cases of the one introduced 
above for MCCGs (recall Section [Tj). Note also a MCCG G represents the same separations 
over V as the AMP CG H obtained by replacing every bidirected edge A <->■ B in G with 
A *- Hab B. Therefore, G represents the marginal independence model of H over V. See 
Section 14.31 for a discussion on the relationship of MCCGs with other families of graphical 
models. 

Note that if a MCCG has a subgraph A - C - B such that C has some spouse, then the 
constraint CI implies that there must be an edge between A and C in the MCCG, whereas 
the constraint C2 implies that the edge must be undirected. Therefore, if a MCCG has a 
path A = V\ - Vi - . . . - V n = B such that Vi has some spouse for all 1 < % < n, then the edge 
Vi - V n must be in the MCCG. Therefore, the independence model induced by a MCCG is 
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the same whether we use the definition of Z-open path above or the following simpler one. 
A path p in a MCCG is said to be Z-open when 

• every triplex node in p is in Z, and 

• every non-triplex node in p is not in Z. 

The theorem below shows that the independence models induced by MCCGs are not arbi- 
trary in the probabilistic framework. 

Theorem 2. For any MCCG G, there exists a regular Gaussian probability distribution p 
that is faithful to G. 

Proof. It suffices to replace every bidirect ed edge A «-> B in G with A <- Hab -*■ B to create 



an AMP CG H, apply Theorem 6.1 by iLevitz et al.l (120011 ) to conclude that there exists 



a regular Gaussian probability distribution q that is faithful to H, and then let p be the 
marginal probability distribution of q over V. □ 

Corollary 1. The independence models induced by MCCGs satisfy the graphoid, composition 
and weak transitivity properties. 

Proof. It follows from Theorem [2] by just noting that the set of inde pendencie s in an y regular 



Gaussian probability distribution satisfy the properties mentioned (IStudenyi . 120051 . Sections 



2.2.2, 2.3.5 and 2.3.6). □ 

Another interesting propert y of MCCGs is th at they are closed under marginalization (a.k.a 
the precollapsibility property ( Studenv . 1997bl )): For every MCCG G and U £ V, there exists 



a so-called marginal MCCG G u such that X± G uY\Z if and only if X L G Y\Z for all X, Y 
and Z disjoint subsets of U. Specifically, G u can be obtained from Gu by adding an edge 
A - B to it if G has a path A - . . .- B such that A and B are the only nodes in the path that 
are in U. 

Finally, we show below that the independence model induced by a MCCG coincides with 
certain closure of certain separations. We define the local separation base of a MCCG G as 
the following set of separations: 

• AlB for all A,B e V such that A and B are not adjacent in G and are in different 
undirected connectivity components of G, and 

• AlB\nec{A) for all A,BeV such that A and B are not adjacent in G and are in the 
same undirected connectivity component of G. 

We define the closure of the local separation base of G, denoted as cl(G), as the set of 
separations that are in the base plus those that can be derived from it by applying the 
graphoid, composition and weak transitivity properties. We denote the separations in cl(G) 
as Xl d{G) Y\Z. 

Theorem 3. For any MCCG G, if X l d(G) Y\Z then Xl G Y\Z. 

Proof. Since the independence model induced by G satisfies the graphoid, composition and 
weak transitivity properties by Corollary [I], it suffices to prove that the local separation base 
of G is a subset of the independence model induced by G. We prove this next. If two non- 
adjacent nodes A and B are not in the same undirected connectivity component of G, then 
every path between A and B in G has some triplex node. Therefore, AiqB. On the other 
hand, if A and B are in the same undirected connectivity component of G, then every path 
between A and B in G falls within one of the following cases. 

Case 1: A = V± - V2 - V3 . . .V n = B such that V2 has no spouse in G. Then, this path is 
not nec(Vi)-open. 

Case 2: A = V± - V2 - ■ ■ ■ - V m - V m+ i - V m+ 2 . ..V n = B such that Vi has some spouse in G 
for all 2 < i < m and V m+ i has no spouse in G. Note that V* e nea(Vi) by constraints 
CI and C2 for all 2 < i < m + 1. Then, this path is not nec(Vi)-open. 
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Case 3: A = Vi - V 2 Vz . . . V n = B. Note that V 3 * V n and V3 i ne G (Vi) by constraint 
C2. Then, V 3 is a triplex node in this path and, thus, this path is not nec(Vi)-open. 

Case 4: A = Vi -V 2 - . . . -V m - V m+ i <-»■ V m+2 ...V n = B such that y has some spouse in 
G for all 2 < % < m. Note that V m+ 2 + V n and V m+ 2 i ne G (Vi) by constraint C2. Then, 
V m+2 is a triplex node in this path and, thus, this path is not nec(Vi)-open. 

Case 5: A = V x ±± V 2 . . . V n = B. Note that V 2 + V n by constraint C2. Then, V 2 is a 
triplex node in this path and, thus, this path is not ne G (Vi)-open. 

Consequently, A± G B\ne G (A). 

□ 

Lemma 8. Let G be a MCCG, A,B eV and Z c y \ {A, B) . If A± G B\Z and a node CeZ 
has some spouse in G, then A± G B\Z \ C . 

Proof. Assume the contrary. Then, there is a path p between A and B in G that is (Z \ C)- 
open. Moreover, C must occur in p because, otherwise, p would also be Z-open which would 
contradict the assumption that ALgB\Z. For the same reason, C must be a non-triplex node 
in p. Let D - C - E be a subpath of p. Note that the edge D - E is in G by definition of 
MCCG, because C has some spouse in G. Then, the path obtained from p by replacing the 
subpath D-C - E with the edge D - E is Z-open. However, this contradicts the assumption 
that A± G B\Z. □ 

Theorem 4. For any MCCG G, if X L G Y\Z then Xl cl(G) Y\Z. 

Proof Since the independence model induced by G satisfies the decomposition property and 
cl(G) satisfies the composition property, it suffices to prove that if A± G B\Z then A±d( G )B\Z 
with A,B e V and Z c y \ {A,B}. We prove this result by induction on \Z\. If \Z\ = 0, 
then A and B must be in different undirected connectivity components of G. Consequently, 
A± c i( G )B. Assume as induction hypothesis that the theorem holds for \Z\ < I. We now prove 
it for \Z\ = l. Consider the following cases. 

Case 1: A and B are in the same undirected connectivity component K of G. 

Case 1.1: All the nodes in Z are in K. Then, A 1 d(G)B\Z (ILauritzenl . [1996, 
Theorem 3.7). 

Case 1.2: There is some node CeZ that is not in K such that C is a spouse of 
some node in K and A / G C\Z \ C. Then, B 1 G C\Z \ C. To see it, assume 
the contrary. Then, A / G C\Z \ C and B / G C\Z \ C imply A / G B\Z \ C or 
AJ_ G B\Z by weak transitivity, which implies AJ_ G B\Z \ C because A± G B\Z by 
assumption. However, this contradicts Lemma [HI 

Finally, note that B 1 G C\Z \ C implies B 1 d i G \C\Z \ C by the induction hy- 
pothesis. Note also that A± G B\Z \ C by Lemma and, thus, A ± d ( G )B\Z \ C 
by the induction hypothesis. Then, A± c ir G -\B\Z by symmetry, composition and 
weak union. 

Case 1.3: Cases 1.1 and 1.2 do not apply. Let C be any node in Z that is not 
in K. Then, A 1 G C\Z \ C. Note also that A 1 G B\Z \ C. To see it, assume 
the contrary. Then, there is a path p between A and B in G that is (Z \ C)- 
open. Moreover, C must occur in p because, otherwise, p would also be Z-open 
which would contradict the assumption that A 1 G B\Z. However, this implies 
that AJ_ G C\Z \ C, which is a contradiction. 

Finally, note that A 1 G C\Z \ C and A 1 G B\Z \ C imply A 1 d (G)C\Z \ C and 
A±d(G)B\Z \ C by the induction hypothesis. Then, A±d( G )B\Z by composition 
and weak union. 

Case 2: ^4 and B are in different undirected connectivity components of G. Let A be 
in the undirected connectivity component K of G. 
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Case 2.1: There is some node C e Z that is a spouse of A. Then, B ]_gC\Z \ C 
and, thus, B ± d (G)C\Z \ C by the induction hypothesis. Note that A± G B\Z \ C 
by Lemma [S] and, thus, A l ci ( G )£>|Z \ C by the induction hypothesis. Then, 
A± c i(g)B\Z by symmetry, composition and weak union. 

Case 2.2: There is some node C e Z that is in K such that C has some spouse in 
G and A± G C\Z \ C. Then, A]_ d ^ G )C\Z \ C by the induction hypothesis. Note 
that ^4 l qB\Z \ C by Lemma [8] and, thus, A l d iQ\B\Z \ C by the induction 
hypothesis. Then, ALmg)B\Z by composition and weak union. 

Case 2.3: There is some node C e Z that is in K such that C has some spouse 
in G and A / G C|Z x C. Then, B l G C|Z \ C. To see it, assume the contrary. 
Then, A / G C\Z \ C and B / G C|Z \ C imply A / G 5|Z \ C or A / G 5|Z by 
weak transitivity, which implies Aj_ G B\Z \ C because Al g B\Z by assumption. 
However, this contradicts Lemma [HI 

Finally, note that B ]_gC\Z\C implies Bi c ug)C\Z\C by the induction hypoth- 
esis. Note also that A±qB\Z \ C by Lemma[8]and, thus, A± d ( G )B\Z \ C by the 
induction hypothesis. Then, A± d ( G )B\Z by composition and weak union. 
Case 2.4: Cases 2.1-2.3 do not apply. Let V%, . . . , V m be the nodes in Z that are in 
K. Let Wi, . . . ,W n be the nodes in Z that are not in K. Then, AlcB, Vl±gB, 
AlaWj, and Vil G Wj for all 1 < i < m and 1 < j < n. Then, Al d(G) i?, Vi±d(G)B, 
A l d( G )W^-, and Vi 1 d(G)Wj for all 1 < i < m and 1 < j < n by the induction 
hypothesis. Then, ALmg)B\Z by symmetry, composition and weak union. 

□ 

Let Q be a partition of V. We say that a MCCG G is consistent with Q if every bidirected 
edge in G has its end nodes in different elements of Q, and every undirected edge in G has 
its end nodes in the same element of Q. Note that the elements of Q may not be undirectly 
connected in G and, thus, they may not coincide with the undirected connectivity components 
of G. Therefore, every undirected connectivity component of G is contained in some element 
of Q but an element of Q may contain several undirected connectivity components of G. 

We define the pairwise separation base of a MCCG G relative to a partition Q of V that 
is consistent with G as the following set of separations: 

• AlB for all A, B e V such that A and B are not adjacent in G and are in different 
elements of Q, and 

• AlB\Q \ {A, B} for all A,B eV such that A and B are not adjacent in G and are in 
the same element Q of Q. 

We define the closure of the pairwise separation base of G relative to Q, denoted as 
cp(G, Q), as the set of separations that are in the base plus those that can be derived from 
it by applying the graphoid, composition and weak transitivity properties. We denote the 
separations in cp(G, Q) as X ± cp (g,q)Y\Z. 

Theorem 5. For any MCCG G and any partition Q of V that is consistent with G, X l 
d(G)Y\Z if and only if X ]_ cp(G ,Q)Y\Z . 

Proof. It suffices to prove that the separations in the local (respectively pairwise) separation 
base are in the closure of the pairwise (respectively local) separation base. 

Let A belong to the element Q of Q. Let A belong to the undirected connectivity component 
K of G. Recall that K £ Q. Let Vi, . . . , V\ denote the nodes in ne G (A). Let Vi+±, . . . , V m 
denote the nodes in K \ ne G (A) \ A. Let V m+ i, . . . , V n denote the nodes in Q \ K. Then, 
A l Vi\Q \ {A, Vi} is in the pairwise separation base of G for all I + 1 < % < n. Then, A l 
{Vi + i, . . . ,V n }\{Vi, . . . ,Vi} is in cp(G, Q) by intersection and, thus, A 1 Vi} is in 

cp(G, Q) by decomposition for all I + 1 < % < n. Consequently, the separations in the local 
separation base are in the closure of the pairwise separation base. 



14 



Likewise, note that A±V^|{Vi, . . . , V/} is in the local separation base of G for allZ+1 < j < m. 
Note also that there is no bidirected edge in G between any two nodes in Q, because G is 
consistent with Q. Therefore, AlVk and VilVk are in the local separation base of G for all 
1 < i < I and m + 1 < k < n. Then, A±{V[ + i, . . . , V m }\{Vi, . . . , V/} is in cl(G) by composition. 
Moreover, A u {V\, . . . , V/} 1 {V m+ i, . . . , V n } is in cl(G) by symmetry and composition and, 
thus, A l {V m+ i, . . . , V^}|{Vi, . . . , Vi} is in cl(G) by symmetry and weak union. Then, A l 
{VJ + i, . . . , . . . , Vi} is in cl(G) by composition and, thus, Ai Vi\Q \ {^4, Vi} is in cl(G) 

by weak union for all I + 1 < i < n. Consequently, the separations in the pairwise separation 
base are in the closure of the local separation base. □ 

Corollary 2. For any MCCG G and any partition QofV that is consistent with G, X ±gY\Z 
if and only ifX± d{G) Y\Z if and only if X ± cp{GjQ) Y\Z . 

4.1. Markov Equivalence of MCCGs. We say that two MCCGs are Markov equivalent if 
they induce the same independence model. In a MCCG, a triplex ({A, C},B) is an induced 
subgraph A B C, A^B-CoiA-B-^C. We say that two MCCGs are triplex 
equivalent if they have the same adjacencies and triplexes. 

Theorem 6. Two MCCGs are Markov equivalent if and only if they are triplex equivalent. 

Proof. We first prove the "only if part. Let G\ and G2 be two Markov equivalent MCCGs. 
First, assume that Gi and G 2 do not have the same adjacencies. Specifically, assume with- 
out loss of generality that A and B are adjacent in G2 but not in G±. Then, A 1 Gl B or 
A 1 dBlned^A) by Theorem [3] but neither of the two separations holds in G 2 , which is a 
contradiction. 

Second, assume that G\ and G2 have the same adjacencies but different triplexes. Specif- 
ically, assume without loss of generality that G± has a triplex ({A, C},B) that G2 does 
not have. Assume also without loss of generality that A B is in G%. Then, A 1 Gl C 
or A 1 G 1 C\nec 1 (A) by Theorem [3] but neither of the two separations holds in G2 because 
B i nea 1 {A), which is a contradiction. 

We now prove the "if part. Let Gi and G2 be two triplex equivalent MCCGs. We prove 
below that X / GiY\Z implies X / q^Y\Z. The opposite implication can be proven in the 
same manner by just exchanging the roles of G\ and G2 in the proof. Specifically, assume 
that X / q^Y\Z. Let p\ be any of the shortest Z-open paths between a node in X and a 
node in Y in G\. Let P2 be the path in G2 that consists of the same nodes as p%. Then, P2 is 
Z-open. To see it, assume the contrary. Then, one of the following cases must occur. 

Case 1: P2 does not have a triplex ({A,C}, B), B e Z, and B has no spouse in G2. 
Then, one of the following cases must occur. 

Case 1.1: p\ has a triplex ({A,C}, B). Then, A and C must be adjacent in G\ 
and G 2 because, otherwise, Gi and G 2 would not be triplex equivalent. Let Qi 
be the path obtained from p\ by replacing the triplex ({A,C}, B) with the edge 
between A and C in G\. Note that Qi cannot be Z-open because, otherwise, it 
would contradict the definition of p\. Then, Qi is not Z-open because A or C do 
not meet the requirements. Assume without loss of generality that C does not 
meet the requirements. Then, one of the following cases must occur. 

Case 1.1.1: Qi does not have a triplex ({A, D},C), C e Z, and C has no 
spouse in G\. Then, one of the following subgraphs must occur in G\. 

A^lh^C — D A^B^C — D A ^—B~^ C — D 

However, the first and third subgraphs imply a contradiction, because C 
has some spouse in G\. The second subgraph also implies a contradiction, 
because pi is not Z-open. 
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Case 1.1.2: qi has a triplex ({A, D},C) and C i Z. Note that C cannot be 
a triplex node in p 1 because, otherwise, p\ would not be Z-open. Then, the 
following subgraph must occur in G±. 

A'^B—^C — D 

Moreover, the subgraph above implies that the edge B-D must be in G\ 
by definition of MCCG. Then, the path obtained from p\ by replacing the 
subpath B-C-D with the edge B-D is Z-open. However, this contradicts 
the definition of p\. 

Case 1.2: pi does not have a triplex ({A, C},B). Then, B must have some spouse 
in G*i, because B e Z and pi is Z-open. Then, the edge A - C must be in Gi by 
definition of MCCG. Then, the path obtained from pi by replacing the subpath 
A-B-C with the edge A - C is Z-open. However, this contradicts the definition 
of pi. 

Case 2: p 2 has a triplex ({A,C}, B) and B i Z. Then, p x does not have a triplex 
({A,C},i?) because, otherwise, pi would not be Z-open. Then, A and C must be 
adjacent in Gi and G2 because these are triplex equivalent. Let £1 be the path 
obtained from pi by replacing the triplex ({A,C},B) with the edge between A and 
C in Gi. Note that £1 cannot be Z-open because, otherwise, it would contradict the 
definition of p\. Then, Qi is not Z-open because A or C do not meet the requirements. 
Assume without loss of generality that C does not meet the requirements. Then, one 
of the following cases must occur. 

Case 2.1: q x does not have a triplex ({A, D}, C), C e Z, and C has no spouse in 
G\. Then, the following subgraph must occur in G\. 

A^B^C — D 

However, this subgraph implies that pi is not Z-open, which is a contradiction. 
Case 2.2: Qi has a triplex ({A, D},C) and C £ Z. Note that C cannot be a triplex 
node in p 1 because, otherwise, p\ would not be Z-open. Then, the following 
subgraph must occur in G\. 

A* :: ^B^ 1 C — D 

Moreover, the subgraph above implies that the edge B-D must be in Gi by 
definition of MCCG. Then, the path obtained from pi by replacing the subpath 
B-C-D with the edge B-D is Z-open. However, this contradicts the definition 
of pi. 

□ 

Lemma 9. For every triplex equivalence class of MCCGs, there is a unique maximal (with 
respect to set inclusion) set of bidirected edges such that some MCCG in the class has exactly 
those bidirected edges. 

Proof. Assume to the contrary that there are two such sets of bidirected edges. Let the 
MCCG G have exactly the bidirected edges in one of the sets, and let the MCCG H have 
exactly the bidirected edges in the other set. For every edge A «-> B in G such that A - B is 
in H, replace A - B with A «-> B in H and call the resulting graph F. We prove below that 
F is a MCCG that is triplex equivalent to G, which is a contradiction since F has a proper 
superset of the bidirected edges in G. 
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First, we show that F has no induced subgraph A - C - B such that C has some spouse 
D in F. Assume the contrary. Then, the induced subgraph A - C - B must occur in G and 
H . Moreover, the edge C D must be in G or H . Then, G or H has an induced subgraph 
A - C - B plus the edge C D, which contradicts the definition of MCCG. 

Second, we show that F has no cycle A - ... - B «-> A. Assume the contrary. Then, the 
subgraph A - ... - B must occur in G and H. Moreover, the edge B «-> A must be in G or 
H. Then, the cycle A - . . . - B A must occur in G or H, which contradicts the definition 
of MCCG. 

Third, note that F has the same adjacencies as G. Fourth, note that all the triplexes in 
G are in F too. Finally, assume to the contrary that F has a triplex {{A, C},B) that G 
does not have (and, thus, nor does H). Then, the subgraph A - B - C must be in G and H . 
However, this implies that the subgraph A - B - C is in F, which is a contradiction. □ 

Note that the theorem above does not hold if the word maximal is replaced by minimal. A 
simple counterexample is the triplex equivalence class that contains the MCCGs A «-> B - C 
and A - B C. 

We say that a MCCG G is blarger than another MCCG H if every bidirected edge in H 
is in G. The lemma above implies that every triplex equivalence class has a distinguished 
member, namely the blargest MCCG in the class. We show below how this distinguished 
member can be obtained from any other member of the class. By bidirecting an undirected 
connectivity component K of a MCCG G, we mean replacing every edge A - B in G such 
that A, B e K with an edge A++ B. Moreover, we say that the bidirecting is feasible if K is 
a complete set. 

Lemma 10. The graph H resulting from performing a feasible bidirecting on a MCCG G is 
a MCCG that is triplex equivalent to G. 

Proof. Let K denote the undirected connectivity component of G that got bidirected. First, 
we show that H has no induced subgraph A - C - B such that C has some spouse D in 
H. Assume the contrary. Then, C,D e K because, otherwise, G would not be a MCCG. 
Therefore, A,B,C,D e K and, thus, the edges A «-> C and C ■«-»■ B must be in H, which is a 
contradiction. 

Second, we show that H has no cycle V± - ... - V n «-> V\. Assume the contrary. Then, 
Vi,V n e K because, otherwise, G would not be a MCCG. Therefore, Vi e K for all 1 < i < n 
and, thus, the edge Vi «-> V^+i must be in H for all 1 < i < n, which is a contradiction. 

Third, note that H has the same adjacencies as G. Fourth, note that all the triplexes in 
G are in H too. Finally, assume to the contrary that H has a triplex ({A,C}, B) that G 
does not have. Then, the induced subgraph A-C - B must be in G and, thus, A,B,C e K. 
However, this implies that K is not a complete set, which is a contradiction. □ 

Lemma 11. If no feasible bidirecting can be performed on a MCCG G, then G is the blargest 
MCCG in its triplex equivalence class. 

Proof. Assume to the contrary that H and not G is the blargest MCCG in the triplex equiv- 
alence class of G. Then, there must exist an edge A - B in G such that the edge A <-> B is in 
H. Let K denote the undirected connectivity component of G such that A, B e K. Note that 
K cannot be a complete set, because no feasible bidirecting can be performed on G. Then, 
G has an induced subgraph V1-V2-V3 with V±, V 2 , V3 e K. Note that H also has an induced 
subgraph V1-V2-V3 because, otherwise, G and H would not be triplex equivalent. Then, G 
must have a subgraph V1-V2-V3- .. .- V n -\ - V n such that V1-V2-V3- .. .- V n -\ V n is 
a subgraph of H. Note that V n - 2 and V n must be adjacent in G and H because, otherwise, 
G and H would not be triplex equivalent. Then, the edge V n -2 - V n (respectively V n - 2 V n ) 
must be in G (respectively H) by definition of MCCG. Likewise, the edge Vi - V n (respec- 
tively Vi V n ) must be in G (respectively H) for all 1 < i < n - 3. However, this implies that 



Table 3. Algorithm for learning MCCGs. 
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Input: A probability distribution p that is faithful to an unknown MCCG G. 






Output: The blargest MCCG H in the triplex equivalent class of G. 




1 

1 


Let H denote the complete bidirected graph 




o 
I 


Set I = 




Q 
O 


Repeat while I < \V\ - 2 




4 


For each ordered pair of nodes A and B in H such that A € ad H (B) and ladif(A) \ Bl > I 


5 


If there is some S c adir(A) \ £? such that |5| = I and A± p .B|iS then 




6 


Set Sab - Sba - S 




7 


Remove the edge A +•> _B from _ff 




8 


Set / = / + 1 




9 


Replace every induced subgraph A <-> B C in -ff such that B 6 5ac with A - 


B-C 


10 


If there is an edge A <-» B in if that violates the constraint CI or C2 then 




11 


Replace the edge A ** B in H with A - B 




12 


Go to line 10 





V\-V n -Vz is an induced subgraph of G whereas V\ V n V3 is an induced subgraph of 
which contradicts the assumption that G and H are triplex equivalent. □ 

Theorem 7. TTie blargest MCCG in a triplex equivalence class of MCCGs can be obtained 
from any member of the class by performing feasible bidirectings until no more can be per- 
formed. 

Proof. It follows from Lemmas [10] and [TTJ □ 

By undirecting a set of bidirected edges in a MCCG, we mean the inverse operation of 
bidirecting an undirected connectivity component of a MCCG. In other words, the result 
of undirecting a set of bidirected edges in a MCCG if is a MCCG G such that the result 
of bidirecting an undirected connectivity component in G is H. Moreover, we say that the 
undirecting is feasible if the corresponding bidirecting is feasible. 

Corollary 3. Any member of a triplex equivalence class of MCCGs can be obtained from any 
other member of the class by performing a sequence of feasible bidirectings and undirectings. 

4.2. Algorithm for Learning MCCGs. In this section, we present a constraint based 
algorithm for learning a MCCG a given probability distribution is fait hful to. The algorit hm, 



which can be seen in Table [3J, resembles the well-known PC algorithm (ISpirtes et al.l . ll993f ). It 
consists of two phases: The first phase (lines 1-8) aims at learning the adjacencies, whereas the 
second phase (lines 9-12) aims at learning the edge type for each adjacency learnt. Specifically, 
the first phase declares that two nodes are adjacent if and only if they are not separated by 
any set of nodes. Note that the algorithm does not test every possible separator (see line 5). 
Note also that the separators tested are tested in increasing order of size (see lines 2, 5 and 
8). The second phase identifies the edge type for each pair of adjacent nodes by avoiding 
false triplexes (line 9) and enforcing the constraints CI and C2 (lines 10-12). 

Theorem 8. After line 12, H is the blargest MCCG in the triplex equivalent class of G. 

Proof. First, we prove that G and H have the same adjacencies after line 8. Consider any 
pair of nodes A and B in G. If A e ad G (B), then Aj. p B\S for all S 9 V \ {A,B} by the 
faithfulness assumption. Consequently, A 6 adn(B) at all times. On the other hand, if 
A £ ada(B), then Al p B or A l p B\neG(A) by the faithfulness assumption and Theorem [31 
Note that, as mentioned before, nec(A) £ adn(A) \ B at all times. Therefore, there will exist 
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some S in line 5 such that A± P B\S and, thus, the edge A <-»■ B will be removed from H in 
line 7. Consequently, A $ adn(B) after line 8. 

Second, G and H must be triplex equivalent after line 12 because, as shown above, they 
have the same adjacencies and lines 9-12 perform only necessary replacements. Actually, for 
the same reason, H must be the blargest MCCG in the triplex equivalent class of G. □ 

4.3. Discussion. In this section, we have introduced a new family of graphical models 
that consists of undirected and bidirected edges. We have named this new family maximal 
covariance-concentration graphs (MCCGs) because it includes both covariance and concen- 
tration graphs as subfamilies. Our motivation for developing MCCGs has been the aim to 
model the covariance and concentration matrices of a Gaussian probability distribution jointly 
by a single graph, rather than modeling the former by a covariance graph and the latter by a 
concentration graph. We have argued that, by doing so, MCCGs may model more accurately 
the probability distribution. We show below an example that illustrates this. 

Example 2. Consider a Gaussian probability distribution p that is faithful to the MCCG G 
below. Recall from Theorem® that such a probability distribution exists. 

A — B 

I 

C^D 
G 

The covariance graph and the concentration graph of p are depicted by the graphs H and 
F below. 

A^B A — B 

1/1 IXI 

C^D C—D 
H F 

Now, note that B± p C\A because B±gC\A. However, BJ_hC\A and B]_pC\A. 

Finally, we briefly describe below four extensions to the work presented in this section that 
we are currently exploring. 

• Despite the example above, we do not discard the possibility that some Gaussian 
probability distributions are modeled more accurately by a covariance graph plus a 
concentration graph than by a MCCG. We would like to study when this occurs, if at 
all. 

• We would like to remove the constraint that MCCGs are simple graphs to allow the 
possibility of having an undirected and a bidirected edge between two nodes. 

• We would like to extend MCCGs with directed edges, so that they can represent the 
result of marginalization and/or conditioning in AMP CGs. 

• The correctness of our learning algorithm lies upon the assumption that p is faithful 
to some MCCG. This is a strong requirement that we would like to weaken, e.g. by 
replacing it with the milder assumption that p satisfies the composition property. 
However, as with AMP CGs (recall Section l3T2|) . the extension of Meek's conjecture 
to MCCGs does not hold, as the example below illustrates. This compromises the 
development of score+search learning algorithms that are correct and efficient under 
the composition property assumption. It is not clear to us whether it also does it for 
constraint based algorithms. This is something we plan to study. 

Example 3. Consider the MCCGs F and H below. 
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A 



A 



B—C—D B—C—D 
F H 



Then, 1(H) = {B l H A\C, B ± H A\{C, D} , B 1 H D\C, B 1 H D\{C, A} , B ± H {A, D}\C) . One 
can easily confirm by using the definition of separation that 1(H) c 1(F). One can also 
confirm by using Corollary [3] that there is no MCCG that is triplex equivalent to F or H . 
Finally, it is obvious that one cannot transform F into H by adding a single edge. 

Finally, it is worth mention ing that MCCGs are related to maximal ancestral graphs 



(IRichardson and Spirtesl . l2002f ) as follows. A maximal ancestral graph (MAG) is a graph 
whose every edge is undirected, directed or bidirected, and that satisfies certain topological 
constraints. Among the topological constraints, only the following is relevant in this paper: 
A MAG cannot have a subgraph of the form A °-> B - C, where the circle represents an 
unspecified end, i.e. an arrow tip or nothing. Due to this constraint, the separation crite- 
ria for MAGs and AMP CGs coincide. Moreover, this constraint clearly implies that not 
every MCCG is a MAG. However, every independence model induced by a MCCG can be 
induced by a MAG, as the following proposition shows. Therefore, in this sense, MCCGs are 
a subfamily of MAGs. 

Proposition 1. Every MCCG can be translated into a Markov equivalent MAG by just re- 
placing every subgraph A <-> B - C by A B <- C . 

Note that the replacement in the proposition above may create new bidirected edges. For 
instance, the MCCG A <-> B-C D gets translated into the MAG A B C <-> D. Despite 
the proposition above, there are cases where a MCCG is a more natural representation of 
the domain at hand than a MAG and, thus, MCCGs are still worth studying. The following 
example illustrates this. 

Example 4. Consider the AMP CG A-^B^C-D-E and call it G. Consider the 
independence model resulting from G by marginalizing out B. This model can be represented 
by the MCCG A C - D - E, or by the MAGs A^C^D^EorA^C^D^E. 
However, the MAGs suggest the existence of the causal relationship C *- D or D -*■ E, 
although neither exists in G. On the other hand, the MCCG does not suggest any causal 
relationship and, thus, it is preferable. 

The reason why MAGs conflict with the original model in the example above is that MAGs 
were introduced to represent the result of marginalization and/or conditioning in directed 
acyclic graphs, not in AMP CGs. As mentioned, we would like to extend MCCGs with di- 
rected edges, so that they can represent the result of marginalization and/or conditioning in 
AMP CGs. Two other families of graphical models that induc e all the independence m odels 
induced by MAGs (and th us by MCCGs) are summary graphs ( Cox and Wermuth . 1996) and 



MC graphs (Koster, 20021 ) . However, these families have a rather co unterintuitive and unde- 



sirab le feature: Not every missing edge corresponds to a separation (IRichardson and Spirted . 



2001 p. 1023). MCCGs and MAGs, on the other hand, do not have this disadvantage (see, 



respectively, Theorem [3] and ( Richardson and Spirtesl . 2002 . Corollary 4.19)) 



5. Identifying (In)Dependencies from MCCGs 

In this section, we present a graphical criterion for reading dependencies from a MCCG 
G of a probability distribution p, under the assumption that G satisfies some topological 
constraints and p satisfies the graphoid properties, weak transitivity and composition. We 
prove that the criterion is sound and complete in certain sense. 

A MCCG of a WTC graphoid p is a MCCG G such that 
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• the edge A •<-» B is not in G only if Al p B, and 

• the edge A- B is not in G only if A± P B\K \ {A, B}, where K denotes the undirected 
connectivity component of G that contains A and B. 

Note that the separation criterion introduced in Section H] is sound and complete for iden- 
tifying independencies in p from G: It is sound in the sense that it only identifies (true) 
independencies in p, and it is complete in the sense it identifies all the independencies in p 
that can be identified by studying G alone. Soundness follows as follows. Recall from Corol- 
lary [2] that the separations identified in G by this graphical criterion correspond with those 
in cp(G, Q) for any partition Q of V that is consistent with G. Specifically, let Q denote the 
undirected connectivity components of G. Then, p satisfies the independencies corresponding 
to the separations in the pairwise separation base of G relative to Q, by definition of G. Thus, 
p satisfies the independencies corresponding to the separations in cp(G, Q), because p is a 
WTC graphoid. Completeness follows from the fact that there are WTC graphoids that are 
faithful to G (Theorem [2] and Corollary [T]) and, thus, p may be one of them (whether p is 
really faithful to G is impossible to know on the sole basis of G). 

Note that every edge in a MCCG of a WTC graphoid does not correspond to a dependence. 
Since this may be undesirable in some cases, we strengthen the definition above as follows. 
A minimal MCCG (MMCCG) of a WTC graphoid p is a MCCG G such that 

• the edge A B is not in G if and only if Al p B, and 

• the edge A - B is not in G if and only if A ± P B\K \ {A, B}, where K denotes the 
undirected connectivity component of G that contains A and B. 

Note that, by Corollary[2j, we can alternatively define that a MCCG (respectively MMCCG) 
of a WTC graphoid p is a MCCG G such that 

• the edge A B is not in G only if (respectively if and only if) AlpB, and 

• the edge A - B is not in G only if (respectively if and only if) A± p B\neG(A). 

An interesting feature of a MMCCG G of a WTC graphoid p is that it allows us to identify 
not only independencies in p as shown above but also dependencies in p as we show below. 
Specifically, we introduce below a sound and complete graphical criterion for identifying 
dependencies in p from G, under the assumption that G has no cycle with both undirected and 
bidirected edges. This assumption implies that the connectivity components of G form a kind 
of tree, as the following example illustrates. The remark below formalizes this observation. 
Note that both covariance and concentrations graphs always satisfy this assumption. 

A^C—D^F 

\/ 
B^E 

Remark 1. Assume that G has no cycle with both undirected and bidirected edges. Let 
K u be any undirected connectivity component of G. Let Kb be any bidirected connectivity 
component of G. Then, K u n Kb contains at most one node. Moreover, if K u n Kb contains 
the node A, then every path between a node in K u and a node in Kb passes through A. 

Given a MMCCG G of a WTC graphoid p, we know that the following dependencies hold 
in p by definition of G: 

• AJ_ p B for every edge A <-> B in G, and 

• AJ_ P B\K \ {A,B} for every edge A - B in G, where K denotes the undirected con- 
nectivity component of G that contains A and B. 

We call these dependencies the dependence base of p. Further dependencies in p can be 
derived from the dependence base via the WTC graphoid properties. For this purpose, we 
rephrase the WTC graphoid properties in their contrapositive form as follows. Symmetry 
YJ_ P X\Z => XJ_ P Y\Z. Decomposition X J_ P Y\Z => Xj_ p YuW\Z. Weak union X J_ P Y\Z u 
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W ^Xl p Y u W\Z. Contraction X J_ P Y u W\Z => X J_ P Y\Z u W v X l p W\Z is problematic 
for deriving new dependencies because it contains a disjunction in the consequent and, thus, 
we split it into two properties: Contractionl X / p Y u W\Z a X l p Y\Z u W => X / P W\Z, 
and contraction2 I/ p 7u W\Z a X l P W\Z => X / p y|Z u W. Likewise, intersection gives 
rise to intersectionl X J_ P Y u W\Z a X l P Y\Z u W => X / P W\Z u Y, and intersection 
X J_ p Yu W\Z a X l p W|Z u y => X / u W. Note that intersectionl and intersection2 
are equivalent and, thus, we refer to them simply as intersection. Similarly, weak transitivity 
gives rise to weak transitivityl X / P K\Z a K / P Y\Z a X l P Y\Z => X / p y|Z u X, and 
weak transitivity2 X / P K\Z a K / p y |Z a X l p y|Z u X => X / p y |Z. Finally, composition 
X / p y u W|Z =► X / p y|Z v Xj_ p W\Z gives rise to compositionl X 1 P Y u W|Z a X ± p Y\Z => 
XJ_ P W\Z, and composition2 X j_ p Y uW\Z a X ± P W\Z ^> X J_ P Y\Z. Since compositionl and 
composition2 are equivalent, we refer to them simply as composition. The independence in 
the antecedent of any of the properties above holds if the corresponding separation holds in 
G. This is the best solution we can hope for because, as shown above, the separation criterion 
is sound and complete for WTC graphoids. Moreover, this solution does not require more 
information than what it is available, namely G or equivalently the dependence base of p. 
We define the WTC graphoid closure of the dependence base of p as the set of dependencies 
that are in the dependence base of p plus those that can be derived from it by applying the 
nine properties above. Note that we can alternatively define t he dependence bas e of p as the 



following dependencies and the results below would still hold (jPena et al.l . 12009k p. 1083): 

• AJ_ p B for every edge A <-»■ B in G, and 

• Aj_ p B\nec{A) \ B for every edge A - B in G. 

We can now introduce our graphical criterion for identifying dependencies in a WTC 
graphoid fro m its MCCG. It i s wor th mentionin g this graphical criterion subsumes those 



developed by Pena et al. ( 2009 ) and Penal ( 2013 ) for reading dependencies from the covari 



ance graph and concentration graph of a WTC graphoid, respectively. 

Definition 1. Let G be the MCCG of a WTC graphoid p. Let X , Y and Z denote three 
disjoint subsets of V . We say that X is joined to Y given Z in a MCCG G, denoted as 
X~qY\Z, if there exist two nodes A e X and B eY such that there exists a single path pa-.b 
between A and B in G that is U -open with Z <^U ^ X uY u Z \ {A, B}. 

Hereinafter, given a node C in a path pa-.b between two nodes A and B in a MCCG, we 
denote by pa-.c the subpath of pa-.b between A and C. 

Remark 2. In Definition [TJ we can assume without loss of generality that A and B are the 
only nodes in pa-.b that are in X and Y, respectively. 

Proof. Let B' + B be closest node to A that is in pa-.b and Y . Then, pa-.b' is the only path 
between A and B' in G that is [/-open. To see it, assume to the contrary that there is a 
second such path Qa-.b 1 - Note that qa-.b 1 u Pb'-.b cannot be [/-open because, otherwise, there 
would be a second path between A and B in G that is [/-open, which is a contradiction. 
Therefore, one of the following cases must occur. 

Case 1: B' is a non-triplex node in Qa-.b' u Pb'-.b and B' e U. However, that B' e U 
together with the fact that pa,b is [/-open imply that either B' is a triplex node in 
Pa-.b or B' is a non-triplex node in pa-b that has some spouse in G. In either case B' 
has some spouse in G and, thus, qa-.b 1 u Pb'-.b is [/-open, which is a contradiction. 

Case 2: B' is a triplex node in qa-.b' u Pb'-.b and B' £ U. However, that B' $ U together 
with the fact that pa-.b is [/-open imply that B' is a non-triplex node in pa-.b- Moreover, 
that B' is a triplex node in qa-.b' u Pb'-.b implies that B' has some spouse in G. Then, 
removing B' from pa,b results in a second path between A and B in G by definition 
of MCCGs which, moreover, is [/-open, which is a contradiction. 
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The proof for A is similar. 

□ 

Remark 3. In Definition [TJ we can assume without loss of generality that the nodes in U that 
are not in Z or pa-.b have no spouse in G. 

Proof. Let C be a node that is in U but not in Z or pa-.b- Assume that C has some spouse in 
G. Then, pa-.b is the only path between A and B in G that is (U\ C)-open. To see it, assume 
to the contrary that there is a second such path qa-.b- Note that C must be a non-triplex node 
in Qa-.b because, otherwise, that path would also be [/-open, which is a contradiction. For 
the same reason, C cannot have any spouse in G. However, this contradicts the assumptions 
made. □ 

Remark 4. In Definition [TJ we can assume without loss of generality that U contains exactly 
the nodes in X u Y u Z that are in Z or pa-.b or that have no spouse in G. 

Proof. By Remark [21 we can assume without loss of generality that the nodes in U that are 
not in Z or pa-.b have no spouse in G. Let CeXuFuZbea node that is not in Z or pa-.b 
and that has no spouse in G. Then, pa-.b is the only path between A and B in G that is 
(U u C)-open, because C can neither activate new paths nor deactivate pa-.b- Repeating this 
reasoning until no such node C exists leads to the desired result. □ 

The following two theorems prove that the graphical criterion defined above is sound and 
complete in some sense. We start by proving some auxiliary results. 

Lemma 12. Let G be a MMCCG of a WTC graphoid p. Let A and B denote two nodes that 
are in the same bidirected connectivity component of G. If A~qB\U , then AJ_ P B\U is in the 
WTC graphoid closure of the dependence base of p. 

Proof. Let if denote the bidirected connectivity component that contains A and B. Let Sa 
denote the nodes in U\K that are in nec(A) or connected to A by a path that passes through 
neo(A). Let Sb denote the nodes in U \ if that are in nec(B) or connected to B by a path 
that passes through nea(B). Let S denote the nodes in U \ if \ Sa x Sb that are connected 
to A or B by a path that passes through spc(A) or spa(B), respectively. 

Note that A ~ gB\U n if. Note also that the path that makes this statement hold only 
contains bidirected edges, because all its nodes are in if by Remark [p Thus, A]_ P B\U n if is 
in the WTC graphoid closure of the dependence base of p f Penal . [20131 . Theorem 5.1). Then, 



A J_ p B u Sb\U n if by decomposition. Moreover, A l gSb\U n if follows from Remark [TJ 
Therefore, A]_ p B\U n K u Sb by contraction2 and A u Sa J- p B\U n K u Sb by decomposition. 
Moreover, Sa±gB\U nKuS B follows from Remark [TJ Therefore, Aj_ p B\U n K uS b uSa by 
symmetry and contraction2. 

Let D be any node in S. Then, one of the following cases must occur. 

Case 1: A 1 gD\U n K u Sb u Sa or B l qD\U n if u 5b u S^. Assume without loss 
of generality that A l G D\U n K u S B u S A - Then, A J_ P B u D\U n K u Sb u Sa by 
decomposition and Aj_ p B\U n If u u S 1 ^ u D by contradiction2. 

Case 2: A / G D|C7 n if u Sb u S a and 5 / G D\U n K u S B u S A - Then, there are two 
paths pa-.d an d Pb:D that are ([/ n if u 5^ u S^-open. Note that p A:D and p^iZ) 
are of the forms A <-> ... <-»• C - . . . D and S ... «-> C - . . . D, respectively, by 
Remark [TJ Note also that pa-.c does not contain B because, otherwise, pa-.d would 
not be (U n if u Sb u £A)-open since i? would be a triplex node in p^:D that is not 
in [/ n if u Sb u S^. Likewise, pb : c does not contain A Now, let C' + C denote the 
closest node to A and B that is in p^ic and Pb-.c- Then, pa-.c u Pc-.b is a path which, 
moreover, is (U nifu^BU 5' J 4)-open. To see the latter, note that C' is a triplex node 
in both pa-.d and pa-.c u Pc*':B and, moreover, it is in U n if u Sb u ^ because pa-.d is 
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(U n if u Sb u 5' y i)-open. However, this implies that there is a second path between A 
and B that is [/-open, which contradicts the assumption that A~gB\U . 

Therefore, by repeating the reasoning above for the rest of the nodes in S, we can conclude 
that A J_ P B\U n if u S B u S A u S. 

Finally, note that A±gU \ K \ Sb ^ Sa ^ S\U n if u Sb u Sa u S because there is no path 
between A and U \ if \ S B \ S A s S. Then, AJ_ p B uU \ K \ Sb x Sa \ S\U n if u Sb u Sa u 5 
by decomposition and AJ_ P B\U by contraction2. 

Note that the above derivation of Aj_ p B\U only made use of the dependencies in dependence 
base of p and the nine properties introduced at the beginning of this section. Thus, A]_ P B\U 
is in the WTC graphoid closure of the dependence base of p. □ 

Lemma 13. Let G be a MMCCG of a WTC graphoid p. Let A and B denote two nodes that 
are in the same undirected connectivity component of G. If A~qB\U , then Aj_ p B\U is in the 
WTC graphoid closure of the dependence base of p. 

Proof. Let if denote the undirected connectivity component that contains A and B. Let 
Sa denote the nodes in U \ if that are in spc(A) or connected to A by a path that passes 
through spc(A). Let Sb denote the nodes in U \ if that are in spc(B) or connected to B 
by a path that passes through spa(B). Let S denote the nodes in U \ if \ Sa^ Sb that are 
connected to A or B by a path that passes through nec(A) or nea(B), respectively. 

Note that A ~ qB\U n if. Note also that the path that makes this statement hold only 
contains undirected edges, because all its nodes are in if by Remark [Tj Thus, _ Aj p B\U n K 



\ I I -I ' r 

is in the WTC graphoid closure of the dependence base of p ( iPena et al.l . 120091 Theorem 5). 



Then, AJ_ p BuSb\U nK by decomposition. Moreover, A±qSb\U nK follows from Remark[TJ 
Therefore, AJ_ P B\U n K u Sb by contraction2 and A u Sa J- p B\U n X u Sb by decomposition. 
Moreover, S^Ig^I^ nKu Sb follows from Remark [TJ Therefore, Aj_ p B\U n if u 5s u Sa by 
symmetry and contraction2. 

Let D be any node in S 1 . Then, one of the following cases must occur. 

Case 1: A ]_ gD\U n K u Sb u 5a or Si n K u Sb u Sa- Assume without loss 

of generality that A l G D\U n if u S B u S A . Then, A / P S u D\U n if u S B u S A by 
decomposition and Aj_ p B\U n if u Sb u Sa u D by contradiction2. 

Case 2: A/ G D|C/n ifu S B u Sa and B}_ G D\Ur\ if u S B u Sa- Then, one of the following 
cases must occur. 

Case 2.1: All the paths between A and D that are (U n if u Sb u SA)-open pass 
through £> or all the paths between B and D that are (U nif uSBuSA)-open pass 
through A. Assume without loss of generality that all the paths between A and 
D that are (U n if u Sb u Sa )-open pass through B. Since B £U nif uSbuSa, -B 
must be a non-triplex node in all these paths. Therefore, none of these paths is 
(U nif uSBuSAui?)-open because, otherwise, B would have to have some spouse 
in G and, thus, removing B from any of these paths would result in a path between 
A and D by definition of MCCGs which, moreover, would be (U n if u Sb u Sa)- 
open and would not pass through B, which contradicts the assumption that 
such a path does not exist. Consequently, A l qD\U n K u Sb u Sa 1 - 1 B. Then, 
Aj_ p B u D\U nif uS B u Sa by decomposition and Aj_ p B\U n if u S B u Sa u D by 
intersection. 

Case 2.2: There are two paths pa-d and pB:B that are (U n if u Sb u Sa )-open and 
such that they do not pass through B and A, respectively. Note that pa.d and 
Pb-.d are of the forms A- . . .-C . . . D and B - . . .-C . . . D, respectively, by 
Remark [TJ Then, one of the following cases must occur. 

Case 2.2.1: C is the only node that is in pa-.c and Pb-.c- Then, pa-.c u Pc-.b is 
a path which, moreover, is (U nifuSsU Sa )-open. To see the latter, note 
that C is a non-triplex node in p^.c u Pc-.b and it has some spouse in G. 
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However, this implies that there is a second path between A and B that is 
[/-open, which contradicts the assumption that A~ G B\U. 
Case 2.2.2: C is not the only node that is in pa-.c and Pb-.c- Then, let 
C + C denote the closest node to A and B that is in pa-.c and Pb-.c- Then, 
PA-.c rL> Pc-.b is a path which, moreover, is (U nKuSs uS^-open. To see the 
latter, note that C is a non-triplex node in both pa-.d and Pa-.c u Pc-.b and, 
moreover, it is not in UdKuSbuSa because pa-.d is (f/nif uSsuS^-open. 
However, this implies that there is a second path between A and B that is 
[/-open, which contradicts the assumption that A~ G B\U. 

Therefore, by repeating the reasoning above for the rest of the nodes in S, we can conclude 
that AJ_ P B\U n K u S B u S A u S. 

Finally, note that Al G U \ K \ Sb n S a s S\U n if u 5b u Sa u 5 because there is no path 
between A and U \ K \ Sb x <Sa \ S 1 . Then, u [/ s if \ S B s S A \ S|[/ n if u S B u S A u 5 

by decomposition and AJ_ p B\U by contraction2. 

Note that the above derivation of AJ_ p B\U only made use of the dependencies in dependence 
base of p and the nine properties introduced at the beginning of this section. Thus, AJ_ p B\U 
is in the WTC graphoid closure of the dependence base of p. □ 

Theorem 9. Let G be a MMCCG of a WTC graphoid p. IfX~ G Y\Z, then X L P Y\Z is in 
the WTC graphoid closure of the dependence base of p. 

Proof. Let pa-.b and U denote the path and the set of nodes that make X~ G Y\Z hold. Then, 
A~ G B\U. We show below that A]_ P B\U, which implies X J_ P Y\Z by symmetry, decomposition 
and weak union. 

Let m denote the number of connectivity components pa-.b passes through. If m = 1, then 
the result holds by Lemma [12] or [13j Assume as induction hypothesis that the result holds 
for all m < n. We now prove it for m = n. Let C denote the farthest node from A that is in 
Pa-.b and in the same connectivity component as A. Note that C e U. Note also that pa-c 
and pc-.B are the only paths between A and C and between C and B that are (U \ C)-open 
because, otherwise, there would be a second path between A and B that is [/-open by Remark 
ffl which contradicts A~ G B\U. Then, A~ G C\U s C and C~ G B\U s C and, thus, AJ. P C\U\C 
and C J- p B\U \ C by the induction hypothesis. Note that A± G B\U \ C by Remark [TJ Then, 
AJ_ p B\U by weak transitivity 1. 

Note that the above derivation of X J_ P Y\Z only made use of the dependencies in depen- 
dence base of p and the nine properties introduced at the beginning of this section. Thus, 
X j_ p Y\Z is in the WTC graphoid closure of the dependence base of p. □ 

Theorem 10. Let G be a MMCCG of a WTC graphoid p. If X I P Y\Z is in the WTC 
graphoid closure of the dependence base of p, then X~ G Y\Z. 

Proof. Clearly, all the dependencies in the dependence base of p are identified by the graphical 
criterion in Definition [TJ Therefore, it only remains to prove that this graphical criterion 
satisfies the nine properties introduced at the beginning of this section. 

• Symmetry Y~ G X\Z => X ~ G Y\Z. The path pa-b and the set of nodes U that make 
the left-hand side hold also make the right-hand side hold. 

• Decomposition X ~ G Y\Z => X ~ G Y u W\Z. The path pa-.b and the set of nodes U 
that make the left-hand side hold also make the right-hand side hold. 

• Weak union X ~ G Y\Z u W => X ~ G Y u W\Z. The path p A:B and the set of nodes U 
that make the left-hand side hold also make the right-hand side hold. 

• Contractionl X~ G Y u W\Z aX± g Y\Zu W => X~ G W\Z. Let p A -.B and U denote the 
path and the set of nodes that make the left-hand side hold. Following Remark [21 we 
can assume without loss of generality that A and B are the only nodes in pa-.b that are 
in X and YW, respectively. Note also that X± G Y\ZW implies that no node in the 
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path pa-.b can be in Y . Then, pa-.b is (U \ y)-open. If there is a second path between 
A and B in G that is (U \ y)-open, then let qa-.b be any of the shortest such paths. 
Then, we can find a node C e W \ U such that pa-.b is (U \ Y u C)-open but qa-.b is 
not. To see it, note that if qa-.b is (U \ y)-open, then it must contain a non-triplex 
node D e U nY because, otherwise, qa-.b would be £7-open, which is a contradiction. 
Moreover, note that X ± G Y\Z uW implies that one of the following cases must occur. 
Case 1: qa-.d contains a triplex node that is not in Z or W. However, this contra- 
dicts the assumption that qa-.b is (17 \ y)-open. 
Case 2: qa-.d contains a non-triplex node that is in Z or U n W. However, this 

contradicts the assumption that qa-.b is (17 \ y)-open. 
Case 3: Cases 1 and 2 do not apply. Then, qa-.d must contain a non-triplex node 
C e W \ U. Clearly, p A -.B is (U \ Y u C)-open. Moreover, adding C to £7 \ Y 
does not activate new paths. That is, if a path ip between two nodes in G is 
not (£7 \ F)-open, then it is not (£7 \ Y u C)-open because, otherwise, C would 
have to be a triplex node in ip and, thus, C would have some spouse in G and, 
thus, removing C from qa-.b would result in a path between A and B in G by 
definition of MCCGs which, moreover, would be (£7\y)-open, which contradicts 
the assumption that qa-.b is one of the shortest such paths. 
Therefore, by repeating the reasoning above we can obtain a set of nodes £7' such 
that Z £ U' £ X uW u Z \ {A, B} and pa-.b is the only path between A and B in G 
that is U'-opeia. Consequently, X~ G W\Z holds. 

• Contraction X ~ G F u W|Z a X l G iy|Z => X - G Y\Z u W. Let p A -. B and C/ denote 
the path and the set of nodes that make the left-hand side hold. Following Remark 
El we can assume without loss of generality that A and B are the only nodes in pa-.b 
that are in X and YW, respectively. Note also that X 1qW\Z implies that no node 
in the path pa-b can be in W. Then, pa-.b is (U u VF)-open. If there is a second path 
qa-.b between A and B in G that is (U u lV)-open, then we can find a node Ce(7n7 
such that pa-b is (17 u W v C)-open but qa-.b is not. To see it, note that if qa-.b is 
(U uH / )-open, then it must contain a triplex node D € W\ U because, otherwise, qa-.b 
would be [/-open, which is a contradiction. Moreover, note that X \_qW\Z implies 
that one of the following cases must occur. 

Case 1: qa-.d contains a non-triplex node that is in Z. However, this contradicts 
the assumption that qa-.b is (U u W)-opeia. 

Case 2: qa-.d contains a triplex node that is not in Z or Y \ XJ. However, this 
contradicts the assumption that qa-.b is (£7 u jy)-open. 

Case 3: Cases 1 and 2 do not apply. Then, qa-.d must contain a triplex node 
C e UnY. Clearly, pa,b is (Uu W \ C)-open. Moreover, removing C from {7uVF 
does not activate new paths. That is, if a path ip between two nodes in G is not 
{U u H/)-open, then it is not (UuW \ C)-open because, otherwise, C would have 
to be a non-triplex node in ip. However, recall that C is a triplex node in Qa-.d- 
Then, C has some spouse in G and, thus, ip would be (£7 u H/)-open, which is a 
contradiction. 

Therefore, by repeating the reasoning above we can obtain a set of nodes U' such 
that ZuW£U'^XuYuWuZ \ {A, B} and pa-.b is the only path between A and 
B in G that is £7'-open. Consequently, X ~ qY\Z u W holds. 

• Intersection X~ G YuW\Z a X ± g Y\Z uW ^ X ~ g W\Z uY. Let p A -. B and U denote 
the path and the set of nodes that make the left-hand side hold. Following Remark [2J 
we can assume without loss of generality that A and B are the only nodes in pa-.b that 
are in X and YW, respectively. Note also that X l G Y\Z u W implies that no node 
in the path pa-.b can be in Y. Then, pa-.b is (U u F)-open. If there is a second path 
between A and B in G that is (£7uy)-open, then let qa-.b be any of the shortest such 



paths. Then, we can find a node C eW \ U such that pa-.b is (U u Y u C)-open but 
is not. To see it, note that if qa-.b is (£7 u Y)-open, then it must contain a triplex 
node D eY \ U because, otherwise, qa-.b would be [/-open, which is a contradiction. 
Moreover, note that X ]_qY\Z uW implies that one of the following cases must occur. 
Case 1: qa-.d contains a triplex node that is not in Z or W. However, this contra- 
dicts the assumption that qa-.b is (U u Y)-open. 
Case 2: qa-.d contains a non-triplex node that is in Z or U n W . However, this 

contradicts the assumption that qa-.b is (U u Y)-open. 
Case 3: Cases 1 and 2 do not apply. Then, qa-.d must contain a non-triplex node 
C e W \ U. Clearly, pa-.b is (U u Y u C)-open. Moreover, adding C to U u Y 
does not activate new paths. That is, if a path </? between two nodes in G is 
not (U u Y)-open, then it is not ([/ u Y u C)-open because, otherwise, C would 
have to be a triplex node in ip and, thus, C would have some spouse in G and, 
thus, removing C from qa-.b would result in a path between A and B in G by 
definition of MCCGs which, moreover, would be (U uY)-open, which contradicts 
the assumption that qa-.b is one of the shortest such paths. 
Therefore, by repeating the reasoning above we can obtain a set of nodes U' such 
that ZuY^U'^XuYuWuZ \ {A, B} and pa-.b is the only path between A and 
B in G that is [/'-open. Consequently, X ~gW\Z uY holds. 
• Weak transitivityl X~ g K|ZaK~ g Y|ZaX± g Y|Z ^ X~ G Y|ZuiC Let p A:K and t/ 
denote the path and the set of nodes that make X~ qK\Z hold. Likewise, let Pk-.b and 
W denote the path and the set of nodes that make K ~ qY\Z hold. We show below 
that the path pa-.k u Pi^s and the set of nodes U uW u K make X ~ G Y\Z u if hold. 
Following Remark El we can assume without loss of generality that A is the only node 
in pa-.k that is in X, and that B is the only node in pk-.b that is in Y. Note also that 
X ±qY\Z implies that pa-.k has no node in Y and pk-.b has no node in X. Following 
Remark |3l we assume without loss of generality that the nodes in U that are not in 
Z or pa-.k have no spouse in G, and that the nodes in W that are not in Z or p K:B 
have no spouse in G. 

First, note that pa-.k is the only path between A and K in G that is (U u lY)-open. 
To see it, note that the nodes that are in both W and pk-.b are also in Z and, thus, 
in U. On the other hand, the nodes that are in W but not in Z or Pk-.b do not have 
any spouse in G and, thus, they cannot activate any new path between A and K in 
G. Likewise, pk-.b is the only path between K and B in G that is (U u VY)-open. 

Second, note that pa-.k u Pk-.b is a path, because K is the only node that is in both 
Pa-.k and Pk-b- To see it, assume the contrary. Specifically, let C + K denote the 
closest node to A and B that is in both pa-.k and Pk-.b- Note that the path pa.c u Pc-.b 
cannot be (U u lY)-open by X l qY\Z. Therefore, one of the following cases must 
occur. 

Case 1: C is a non-triplex node in pa-.c u Pc-.b and C e Z. However, that C e Z 
together with the fact that pa-.k is (U u tY)-open imply that C is a triplex node 
in pa-.k- Thus, C has some spouse in G and, thus, pa-.c u Pc*:B is (U u lY)-open, 
which is a contradiction. 

Case 2: C is a triplex node in pa-.c u Pc-.b and C £ Z. However, that C £ Z together 
with the fact that pa-.k is (U u VY)-open imply that C is a non-triplex node in 
Moreover, that C is a triplex node in pa-.c u Pc:B implies that C has some 
spouse in G. Then, removing C from pa-.k results in a second path between A 
and K by definition of MCCGs which, moreover, is (U u VY)-open, which is a 
contradiction. 

Moreover, note that pa-.k u Pk-.b must be ([/ u W u iT)-open because, otherwise, K 
would have to be a non-triplex node in pa-.k^ Pk-.b-, which would contradict X LqY\Z. 
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Finally, if there is a second path between A and B in G that is (U u W u if )-open, 
then if must be a triplex node in that path because, otherwise, that path would 
contradict X l G Y\Z. However, this implies that there is a second path between A 
and K or between K and in G that is (U u IY)-open, which is a contradiction. 

• Weak transitivity2 X ~ G K\Z a K ~ G Y\Z a X ± G Y\Z u K ^ X ~ G Y\Z. Let p A:K and 
[/ denote the path and the set of nodes that make X~ G K\Z hold. Likewise, let pk-.b 
and W denote the path and the set of nodes that make K ~ G Y\Z hold. We show 
below that the path pa-.k u Pk-.b and the set of nodes U uW make X ~ qY\Z hold. 
Following Remark EJ we can assume without loss of generality that A is the only node 
in pa-.k that is in X, and that I? is the only node in pk-.b that is in Y. Note also 
that X l G Y\Z u if implies that PaiK" has no node in Y and has no node in X. 
Following Remark [31 we assume without loss of generality that the nodes in U that 
are not in Z or pa-.k have no spouse in G, and that the nodes in W that are not in Z 
or Pk-.b have no spouse in G. 

First, note that pa-.k is the only path between A and K in G that is (C/u VF)-open, 
and that p,fo.B is the only path between if and B m G that is ([/ u VF)-open. To see 
it, repeat the reasoning in weak transitivity 1. 

Second, note that pa-.k u Pk-.b is a path between A and B in G. To see it, repeat 
the reasoning in weak transitivityl (note that X l G Y\Z u K should be used instead 
of X± G Y\Z). 

Moreover, note that pa-.k u Pk-.b must be (U uW)-opeia because, otherwise, K would 
have to be a triplex node in pa-.k u Pk-.b, which would contradict X l G Y\Z u K. 

Finally, if there is a second path between A and B in G that is (U u W)-open, 
then K must be a non-triplex node in that path because, otherwise, that path would 
contradict X l G Y\Z u K. However, this implies that there is a second path between 
A and K or between K and B in G that is ([/ u IY)-open, which is a contradiction. 

• Composition X~ G Y uW\Z aX ± g Y\Z => X~ G W\Z. Let pa-.b and U denote the path 
and the set of nodes that make the left-hand side hold. Following Remark [21 we can 
assume without loss of generality that A and B are the only nodes in pa-.b that are in 
X and YW, respectively. Note also that X l G Y\Z implies that no node in the path 
Pa-.b can be in Y. Then, pa-b is (U \ y)-open. If there is a second path qa-.b between 
A and B in G that is (U \ y)-open, then we can find a node C e U \ Y such that 

is (J7 \ Y \ C)-open but is not. To see it, note that if qa-.b is (U \ Y)-open, 
then it must contain a non-triplex node D e U n Y because, otherwise, would be 
[/-open, which is a contradiction. Moreover, note that X l G Y\Z implies that one of 
the following cases must occur. 

Case 1: qa-.d contains a non-triplex node that is in Z. However, this contradicts 

the assumption that qa-.b is (U \ Y)-open. 
Case 2: qa-.d contains a triplex node that is not in Z or W \ U. However, this 

contradicts the assumption that qa-.b is (U \ Y)-open. 
Case 3: Cases 1 and 2 do not apply. Then, qa-.d must contain a triplex node 
C 6 U n W . Clearly, pa-.b is (U \ Y \ C)-open. Moreover, removing C from U \ Y 
does not activate new paths. That is, if a path <p between two nodes in G is not 
(U \ Y)-open, then it is not (U \ Y \ C)-open because, otherwise, C would have 
to be a non-triplex node in (p. However, recall that C is a triplex node in qa-.d- 
Then, C has some spouse in G and, thus, ip would be (U \ Y)-open, which is a 
contradiction. 

Therefore, by repeating the reasoning above we can obtain a set of nodes U' such 
that Z ^ U' ^ X uW u Z \ {A, B} and pa-.b is the only path between A and B in G 
that is f/'-open. Consequently, X~ G W\Z holds. 



□ 
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While Theorem |9] may be somewhat expected because if there is a single path between A 
and B in G that is U-open then there is no possibility of path cancelation, the combination 
of Theorems M and [TU] is rather exciting: We now have a simple graphical criterion to decide 
whether a given dependence is or is not in the WTC graphoid closure of the dependence base 
of p, i.e. we do not need to try to find a derivation of it, which is usually a tedious task. 

Corollary 4. Let G be a MMCCG of a WTC graphoid p. Then, X ~ G Y\Z if and only if 
X j_ G Y\Z is in the WTC graphoid closure of the dependence base of p. 

It is worth mentioning that the graphical criterion in Definition [T] is not complete in the 
sense of identifying all the dependencies that are shared by all the WTC graphoids whose 
MMCCG is G. Note also that neither the graphical criterion in Definition Q] nor any other 
sound graphical cr i terion can be complete in the sense of identifying all the dependencies in 
p. See flPena et al.l . I2009L pp. 1082-1083) and flPenal . 1201,1 pp. 202-203) for counterexamples. 



One of the reasons for developing the graphical criterion in Definition [T] is that X J_ G Y\Z 
does not imply X J_ p Y\Z. However, if G has no cycle, then the corollary below proves that 
X J_gY\Z does imply X J_ P Y\Z and, moreover, that this way of identifying dependencies in 
p is sound and complete in the strictest sense possible, since all and only all of them are 
identified. 

Corollary 5. Let G be a MMCCG of a WTC graphoid p. If G has no cycle, then p is faithful 
toG. 

Proof. Assume to the contrary that p is not faithful to G. Since G is a MCCG of p, this 
assumption is equivalent to assume that there exist three disjoint subsets of V, here denoted 
as X, Y and Z, such that X J_ G Y\Z but X l p Y\Z. However, X J_ G Y\Z implies that there must 
exist a path in G between some node A e X and some node B eY that is (XuY uZ\ {A, B})- 
open. Furthermore, since G has no cycle, that must be the only such path between A and 
B in G. However, this implies X ~ gY\Z and thus X / P Y\Z by Theorem |9j which is a 
contradiction. □ 

5.1. Discussion. In this section, we have introduced a sound and complete graphical crite- 
rion for reading dependencies from a MCCG of a WTC graphoid, e.g. a Gaussian probability 
distribution. Recall that our motivation for developing MCCGs has been the aim to model 
the covariance and concentration matrices of a Gaussian probability distribution jointly with 
a single graph, rather than modeling the former with a covariance graph and the latter with a 
concentration graph. We have argued that, by doing so, MCCGs may model more accurately 
the probability distribution. We show below two examples that illustrate this. Specifically, 
the examples show that a MMCCG of a WTC graphoid p can identify more (in) dependencies 
in p than the covariance graph and the concentration graph of p jointly. 

Example 5. Consider a Gaussian probability distribution p that is faithful to the MCCG G 
below. Recall from Theorem that such a probability distribution exists. Note that G is a 
MMCCG of p. 

A — B 

I 

C^D 
G 

The covariance graph and the concentration graph of p are depicted by the graphs H and 
F below. 

A^B A — B 

1/1 IXI 

C^D C—D 
H F 
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Now, note that B± P C\A because B±gC\A. However, BJ_hC\A and B]_pC\A. 

Example 6. Consider a Gaussian probability distribution p that is faithful to the MCCG G 
below. Recall from Theorem [H that such a probability distribution exists. Note that G is a 
MMCCG of p and, moreover, that it has no cycle with both undirected and bidirected edges. 

A. E 

>o 

B D 

G 



The covariance graph and the concentration graph of p are depicted by the graphs H and 
F below. 



B D a ^ 

H F 



E 
D 



Now, note that A / P D\BC because A ~ gD\BC. However, neither A ~ hD\BC nor A ~ 
F D\BC hold. 

Despite the examples above, we do not discard the possibility that some Gaussian proba- 
bility distributions are modeled more accurately by a covariance graph plus a concentration 
graph than by a MCCG. We would like to study when this occurs, if at all. 
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